A Trajectory Based Video Indexing System For Street Surveillance

We have developed a system to analyze and index surveillance videos of a street scene based on the motions of objects in the scene, such as cars and people. A segmentation and tracking system extracts trajectories from compressed video, which are represented in a multiresolution manner and stored in a database. Once the motion segmentation has been accomplished, the three dimensional volume of data representing the object moving through time can be used to robustly estimate other features of objects, including size and color. Hand-drawn queries can be submitted to the system for imprecise searches. The system was tested on 20 minutes of real video footage with numerous moving objects; the success rates for tracking and recall in most cases exceeded 70 percent. Larger, more rigidly moving objects such as cars were more reliably dealt with than smaller objects such as people.